Skip to content

[DRAFT] FEAT Add Agentic Prompt-Injection Boundary Pairs dataset loader#2175

Draft
3nesdeniz wants to merge 1 commit into
microsoft:mainfrom
3nesdeniz:feat/agentic-prompt-injection-boundary-dataset
Draft

[DRAFT] FEAT Add Agentic Prompt-Injection Boundary Pairs dataset loader#2175
3nesdeniz wants to merge 1 commit into
microsoft:mainfrom
3nesdeniz:feat/agentic-prompt-injection-boundary-dataset

Conversation

@3nesdeniz

Copy link
Copy Markdown

Description

Closes #2174.

Adds a remote loader for the CC BY 4.0 Agentic Prompt-Injection Boundary Pairs dataset:

  • 1,200 English records organized as 600 controlled benign/attack pairs;
  • 50 scenario-isolated enterprise and agentic workflows;
  • 12 prompt-injection and trust-boundary attack families;
  • train, validation, test, or combined loading;
  • attack-only default plus benign and full-pair modes;
  • typed filters for label, split, attack family, and source context;
  • pair, scenario, source, boundary, expected-action, and provenance metadata on each seed.

The loader maps each row to a literal SeedPrompt. Family filtering uses pair_family rather than attack_family, which preserves the benign side of a pair when users request the complete boundary surface.

Dataset: https://huggingface.co/datasets/3nesdeniz/agentic-prompt-injection-boundary-pairs

Source and release pipeline: https://github.com/3nesdeniz/agentic-prompt-injection-boundary-pairs

I am the dataset author and maintain the tagged source release. The dataset is synthetic, manually reviewed, and contains no customer or private data.

Tests and Documentation

  • Added 15 focused loader tests covering defaults, all splits, benign-only loading, pair reconstruction, family and source-context filters, invalid enums, empty results, malformed rows, and split conflicts.
  • Ran the full dataset unit-test package: 994 passed.
  • Ran focused loader and metadata coverage tests: 80 passed.
  • Ran all changed files through pre-commit, including Ruff, ty, notebook sanitation, documentation validation, private-key detection, and JSON checks.
  • Performed a live Hugging Face sanity check:
    • test split attack mode: 120 seeds;
    • indirect-content-injection paired mode: 20 seeds / 10 complete pairs;
    • both labels preserved and prompt-group aliases reconstructed.
  • Added the loader to dataset discovery, the loading guide and notebook, and the project bibliography.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FEAT Add Agentic Prompt-Injection Boundary Pairs dataset loader

1 participant